Before version 3 of the Microsoft Windows operating environment was released, the primary operating system used by so-called "PC-compatible" personal computers is the Microsoft Disk Operating System, commonly known as "MS-DOS" or "DOS." MS-DOS has been in existence, in various forms, since approximately 1981. MS-DOS was originally designed to run on computers having the Intel 8088 microprocessor, present in the original IBM PC.
The 8088 microprocessor is able to access one megabyte of memory. However, because the 8088 is a 16-bit microprocessor, it is only able to directly address 64K at a time using one 16-bit register. In consequence, the full megabyte of memory is divided into sixteen 64K segments. A program running on an 8088 can use the value of another 16-bit register to select the segment on which to operate. This is accomplished by forming a 20-bit address from two 16-bit words, represented as segment:offset, according to a scheme that is well known in the art.
These attributes of the 8088 had a direct effect on the design of the original IBM PC. Of the 1 Mb of addressable memory, the first 640K were allocated to random access memory (RAM). The remaining 384K were reserved for use by hardware adapter interfaces, video card frame buffers, and read-only memory (ROM). At the time, 640K was considered to be far more memory than anyone would need.
Microprocessor design has progressed significantly since then, but MS-DOS has not. The architecture of modern personal computers, including the design of MS-DOS, has been driven largely by the desire to retain a certain degree of software and hardware compatibility with earlier models, even the original IBM PC. Accordingly, a typical PC-compatible computer still has 640K of RAM, followed by an area (e.g., 384K) for hardware adapters, video frame buffers, and ROM. The desire for more memory has been accommodated by adding extended or expanded memory after the one megabyte boundary imposed by the 8088 microprocessor. Without these extensions and alternative memory models and addressing schemes, MS-DOS can not use the memory beyond one megabyte to execute programs.
To retain compatibility with earlier computers, later Intel microprocessors, such as the 80286, 80386, 80486, and Pentium, feature a mode known as "real mode," in which the 8088 microprocessor is emulated to allow MS-DOS programs to run. Many MS-DOS programs, TSRs and device drivers, still run or expect to run in real mode. However, these newer microprocessors also offer "protected mode," which allows for a word size of 32 bits and an address space of up to 4 megabytes (or 4,096 megabytes). Programs written in protected mode can treat the entire available address space as a single segment; the entire space is directly addressable, if desired. Protected mode is so called because it also provides three levels of security or "rings" in which programs can be run, with ring 0 the most privileged and ring 3 the least. This technology can be used to help prevent different programs residing in memory from interfering with each other.
Microsoft Windows version 3 runs on PC-compatible computers in conjunction with MS-DOS. MS-DOS (or some other compatible disk operating system) is required to run Windows. Although Windows version 3 and later versions provide the ability to execute Windows programs in protected mode on 80386 and later microprocessors, Windows still relies on real mode MS-DOS for a variety of reasons.
There are many existing items of software designed for the real mode DOS environment that can be useful to a Windows program, including MS-DOS itself. In order to communicate with such real mode software, Windows is typically required to pass data in a form addressable by the real mode software. Similarly, the real mode software must make its data available to Windows at a location real mode software can reach. Consequently, such data can only be passed between Windows and a real mode program through memory space in the first megabyte.
Furthermore, certain hardware devices require memory to be reserved in the first megabyte, typically for communication with real mode device drivers. There may be Windows programs or drivers provided to take control of the device for the duration of the Windows session in order to provide device services to Windows programs. Accordingly, space in the first megabyte may be required for this purpose.
Moreover, due to the fact that Windows runs over MS-DOS, in order to communicate with the real mode DOS layer, certain Windows system components must reserve for shared use portions of memory space in the first megabyte. For example, when a DOS program is executed, it is assigned an area of memory termed the Program Segment Prefix (PSP) in which MS-DOS maintains certain information, including the open files maintained by the program. Real mode MS-DOS implicitly allocates the PSP in the first megabyte of memory. For Windows to interact correctly with MS-DOS, it too must create a PSP in the first megabyte for any program it runs.
When Windows is not running, all memory below 640K and certain memory between 640K and 1M is under the management of DOS. Some memory between 640K and 1M is in the form of Upper Memory Blocks ("UMBs") created by an Expanded Memory Manager such as EMM386 which is provided with both MS-DOS and Windows. UMBs are regions of RAM, possibly mapped in from physical addresses above 1M, located in regions between reserved areas above 640K (such as the video frame buffers and ROM areas previously discussed). Once RAM is mapped in, UMBs can contain real mode programs and device drivers, and also can be a potentially significant source of free space below the 1M boundary.
When Windows is executed, ownership of all DOS memory below 1M is transferred to the Windows Virtual Machine Manager (VMM), a low-level service of Microsoft Windows. This memory is made available to be allocated by virtual device drivers through the .sub.-- Allocate.sub.-- Global.sub.-- V86.sub.-- Data.sub.-- Area service of the VMM. Typically, a small amount of the memory below 640K is used by various system and third-party virtual device drivers. However, since UMB memory must be specifically requested by virtual device drivers, and because it is not available on all computers, it is seldom used. Such unused memory remains unused while Windows runs.
The remainder of the memory below 640K that has not been allocated by a virtual device is provided to the Windows KERNEL to become part of the global heap. The global heap also contains memory from above the 1M boundary. The global heap does not, however, include UMB memory. The portion of the global heap below 1M is known as global DOS memory.
Although special provisions are made by the Windows KERNEL to allow global DOS memory to be specifically requested, it is also treated as an ordinary part of the global heap, subject to allocation by Windows programs and drivers that do not require it. The global heap is a pool of linear address space consisting of global DOS memory below 640K obtained from DOS and memory above 1M obtained from the VMM. Memory can be allocated from the global heap by calling two KERNEL functions: the GlobalAlloc function, which attempts to allocate global heap memory based on several criteria as discussed below; and the GlobalDosAlloc function which attempts to allocate memory from the global DOS portion of the heap.
In attempting to allocate a block of memory from the global heap by calling GlobalAlloc, the calling program must indicate the type of memory requested: fixed, movable, or discardable. A fixed memory block is locked at its allocated linear address within the global heap; it is guaranteed not to move. A movable block can be shifted by the KERNEL in order to satisfy other allocation requests or to reduce heap fragmentation; its position is tracked by the KERNEL. A discardable block is movable, but can also be discarded by the KERNEL in order to increase free space or decrease the size of the global heap. All objects allocated by GlobalAlloc come out of a location within the global heap determined by the Windows KERNEL; there is no mechanism by which an application can request that memory come from a specific part of the heap.
However, by design, the KERNEL tends to allocate discardable and movable blocks from the top part of the global heap and fixed blocks from the bottom part. A "first fit" strategy is used: the KERNEL starts at the top or bottom of the global heap, depending on the allocation type, and moves down or up, respectively, until a free region is found that would accommodate the allocation request; that location is used. Consequently, fixed and movable blocks are generally segregated, alleviating somewhat the problem of heap fragmentation.
Because the bottom part of the global heap consists of global DOS memory, it is likely that any GlobalAlloc request for fixed memory will result in the allocation of fixed global DOS memory. The KERNEL will attempt to locate a fixed object as low as possible, so any movable objects in the way will be relocated to higher memory. As discussed above, once a fixed object is allocated in global DOS memory, there it will remain. This is a potentially undesirable result if the fixed object does not need to be in global DOS memory.
Furthermore, certain KERNEL functions for "locking" movable memory blocks, namely GlobalFix, GlobalWire, and GlobalPageLock, all cause the memory sought to be locked to be relocated to the lowest possible position in the global heap and fixed there until unlocked. As discussed above, this can result in unnecessary use of global DOS memory. Furthermore, other locking functions, such as GlobalLock and LockSegment, will cause a block to be fixed wherever it is within the global heap; if it is already in global DOS memory, it will be fixed there.
Moreover, programs or driver segments loaded by Windows can include a "fixed" attribute, specifying that the segments should be allocated as fixed memory. As discussed above, fixed memory is located low in the global heap, potentially in the global DOS region. The Windows system components that load early in the boot process also tend to be loaded into global DOS memory (because of the first-fit strategy and because the VMM has not yet brought the area beyond 1M into the global heap); these components may then lock themselves wherever they are loaded.
As seen above, numerous factors contribute to the overuse of global DOS memory. However, as cited above, a Windows program or Windows itself is often required to acquire global DOS memory for a valid use. The foregoing factors can lead to a shortage of global DOS memory, causing program failure or the inability to execute programs despite otherwise abundant memory resources.
The 1MBFort program described in the Mar. 28, 1995 issue of PC Magazine represents one attempt to resolve these problems. As discussed, one valid Windows use for global DOS memory is to store the PSPs for executing programs. The solution provided by 1MBFort intentionally fragments the global DOS portion of the global heap into 10K pieces, thereby forcing out movable memory blocks. Because PSP blocks are only 512 bytes long, they are always likely to find room within such a fragmented memory space. However, larger objects are not.
This approach has several drawbacks. Fragmenting the global DOS portion of the heap prevents valid requests for global DOS memory blocks larger than 10K from succeeding. Furthermore, because 1MBFort is an application loaded after the Windows boot process is complete, it cannot affect blocks that are locked into global DOS memory during the boot process. Finally, excessive fragmentation of the global DOS memory space effectively reduces the amount of memory available for bona fide uses.
The Fix1MB program described in the May 1995 issue of Microsoft Systems Journal represents another attempt to minimize the recognized problems. The Fix1MB program recognizes that application programs generally do not allocate or frequently use fixed memory blocks. In order to obtain fixed memory, an application program must explicitly fix the memory using GlobalFix or another of the locking functions previously discussed. The Fix1MB program assumes that the majority of fixed memory allocations occur when an application program or library having the "fixed" attribute, discussed above, is loaded. The Fix1MB program intercepts the LoadModule function, allocates all of the global DOS memory area to itself (less the amount required for the PSP), and holds the global DOS memory until the program or library is loaded. Accordingly, any fixed code segments in the program or library are prevented from consuming global DOS memory, while the PSP can still be allocated as required.
The Fix1MB program has several drawbacks. During the processing of the LoadModule function, the loaded program or library executes its initialization code. If that initialization code contains a valid request for global DOS memory, then the initialization will fail or, if the request can be satisfied out of the area reserved by Fix1MB for the PSP, then a later request to allocate the PSP will fail. Alternatively, if the initialization code launches a second program, then the second request for a PSP might fail. If the loaded module contains a fixed segment the same size or smaller than a PSP, then that segment can be fixed in global DOS memory to the exclusion of the PSP. Finally, the program can, after it is loaded, freely allocate and fix blocks in global DOS memory.
As seen above, prior attempts to resolve problems in over-allocation of global DOS memory have significant drawbacks.